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Scientific  Officer  Code;  1142CS 
Dr.  Susan  E.  Chipman 
Office  of  Naval  Research 
800  N.  Quincy  St. 

Arlington,  VA  22217-5000 

Dear  Susan; 

This  letter  is  a  quarterly  progess  report  for  the  On-Line  Assessment  of  Expertise  project 
(Grant  N0001 4-91 -J-1 532).  It  covers  the  period  from  October,  1991  to  December,  1991. 
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A  great  deal  of  progress  was  made  during  this  period,  in  part  because  Joel  Martin,  arrived  in 
September  and  began  working  on  Olae.  (The  program  and  the  project  are  both  an  accronym  for 
On-line  Assessment  of  Expertise.)  Joel  got  his  dissertation  under  the  supervision  of  Dorritt 
Billman.  Although  his  degree  is  in  Artificial  Intelligence,  he  has  a  great  deal  of  psychological 
backgroun^cluding  the  desj^^^ecution  and  analysis  of  several  experiments.  ^ 

As  you  may  recall,  Olae  (the  program)  has  two  major  parts,  a  user  interface  and  a  data 
analyzer.  The  user  interface  is  nearing  completion.  \Ne  have  rudimentary  versions  of  all  6 
tasks  (problem  solving,  example  studying,  problem  classification,  difficulty  estimation,  basic 
approach,  and  conceptual  problem  solving).  Although  a  pilot  subject  was  run  on  the  problem 
solving  interface,  and  several  important  difficulties  were  uncovered,  we  still  need  to  run  more 

pilot  subjects  in  order  to  make  the  interface  smooth  and  friendly.  _ 
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Most  of  the  progress  v/as  on  the  development  of  the  data  analyzer.  We  settled  on  a  basic  design. 
There  v.ill  be  two  major  components.  One  is  a  Bayesian  network  builder  and  manager.  Given  a 
problem  solving  program  written  in  a  dialect  of  Prolog,  it  will  solve  each  of  the  problems  and 
build  a  network  that  will  analyze  subject  responses  to  those  problems  in  terms  of  the  rules  they 
know.  Joe!  implemented  most  of  this  module  during  this  quarter.  The  Cascade  model  of  physics 
’{problems  solving  has  been  partially  ported  to  the  new  rule-based  language,  and  some  small 
networks  have  been  generated.  How  it  will  scale  to  handle  larger  networks  remains  to  be  seen. 
The  second  component  of  the  data  analyzer,  which  will  be  built  later  after  some  data  is  collected, 
will  interface  the  data  from  the  user  interfaces  to  the  Bayesian  networks. 

I  The  Bayesian  network  package  that  Joel  wrote  turned  out  to  be  rather  fast.  He  added  the  same 
'  sort  of  graphical  editing  and  viewing  features  as  are  found  in  the  leading  commercial  Bayesian 


4  or> 


92-08703 


III  nil 


LRDC  BLDG.,  3939  O'HARA  STREET,  PITTSBURGH,  PA  15ZOU 
TELEPHONE:  (412)  624-7020  FAX:  (412)  624-9149 


I 


network  program,  Hugin,  which  is  used  by  our  collaborators  at  ETS  and  Berkeley.  However, 
because  our  package  is  written  in  Lisp,  we  are  able  to  more  easily  interface  it  with  other  parts 
of  the  system,  including  the  rule-based  language  for  modelling  problem  solving.  Several  people 
have  expressed  interest  in  Joel's  package,  so  we  may  release  it  to  the  general  network 
community  in  the  spring,  after  the  bugs  are  shaken  out  of  it.  In  particular,  we  are  discussing 
the  application  of  this  package  to  Sherlock. 


Sincerely, 


Kurt  A.  VanLehn 

Associate  Prof.  Computer  Science 
Senior  Scientist,  LRDC 


Copies: 

Administrative  Grants  Officer 
Office  of  Naval  Research 
Resident  Representative  N66005 
Administrative  Contracting  Officer 
The  Ohio  State  Univ.  Research  Center 
1314  Kinnear  Rd. 

Columbus,  OH  43212-1194 


Director,  Naval  Research  Laboratory 
Attn:  Code  2627 
Washington,  DC  20375 

Defense  Technical  Information  Center 
Building  5,  Cameron  Station 
Alexandria,  VA  22304-6145 


Statement  A  per  telecon 

Dr.  Susan  Chipman  ONR/Code  H42 

Arlington,  VA  22217-5000 
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